{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "26ed71ee",
   "metadata": {},
   "source": [
    "## Pandas-数据清理与准备总结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7162c50",
   "metadata": {},
   "source": [
    "内容介绍:前面课程的总结。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9da96971",
   "metadata": {},
   "source": [
    "### 数据不规范的表现及主要处理方式\n",
    "\n",
    "* 如果全部在原始数据中进行处理操作的情况下，注意不同操作顺序会改变行序号导致后续操作不能执行"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b25b334",
   "metadata": {},
   "source": [
    " * 部分数据缺失，存在大量的空值。处理方式：\n",
    " \n",
    "找到：DataFrame.isnull()函数找到缺失值。(DataFrame.isnull()).sum()获取存在缺失值列的数量汇总。<br>\n",
    "填充:DataFrame.fillna()<br>\n",
    "删除：DataFrame.dropna(subset='列名',replace=Ture)。删除某列的缺失值。<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f4ae855",
   "metadata": {},
   "source": [
    " * 数据存在重复值\n",
    "\n",
    "找到：DataFrame.duplicated()返回缺失值的Series序列。DataFrame.duplicated().sum()查看缺失值的数量。<br>\n",
    "删除：DataFrame.drop_duplicates(inplace=True)，在元数据上伤处重复数据。<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5c90932",
   "metadata": {},
   "source": [
    " * 部分数据包含数值和字符串，数据类型不统一。字符串的处理方式：\n",
    "\n",
    "检测字符串:DataFrame.[某列名].str.contains()，查看是否包含某字符串。<br>\n",
    "字符串替换：DataFrame.[某列名].map(lambda x:x.replace('万',''))<br>\n",
    "字符串替换(第二种方法)：DataFrame.[某列名].map(lambda x:float(x.replace('万','')))  ## 使用python的float函数转换<br>\n",
    "数字类型转换：DataFrame.[某列名].map(lambda x:x.replace('万','')).astype(np.float32) ##使用pandas的astype函数转换<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e5de86d",
   "metadata": {},
   "source": [
    " * 部分数据存在异常值\n",
    "\n",
    "检测异常值的行：<br>\n",
    "删除异常值的行：Dataframe.drop([行坐标值],replace=True)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "048a3aa0",
   "metadata": {},
   "source": [
    " * 少数数据不利于分析"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
